在这项工作中，我们探讨了随机梯度下降（SGD）训练的深神经网络的限制动态。如前所述，长时间的性能融合，网络继续通过参数空间通过一个异常扩散的过程，其中距离在具有非活动指数的梯度更新的数量中增加距离。我们揭示了优化的超公数，梯度噪声结构之间的复杂相互作用，以及在训练结束时解释这种异常扩散的Hessian矩阵。为了构建这种理解，我们首先为SGD推导出一个连续时间模型，具有有限的学习速率和批量尺寸，作为欠下的Langevin方程。我们在线性回归中研究了这个方程，我们可以为参数的相位空间动态和它们的瞬时速度来得出精确的分析表达式，从初始化到实用性。使用Fokker-Planck方程，我们表明驾驶这些动态的关键成分不是原始的训练损失，而是修改的损失的组合，其隐含地规则地规范速度和概率电流，这导致相位空间中的振荡。我们在ImageNet培训的Reset-18模型的动态中确定了这种理论的定性和定量预测。通过统计物理的镜头，我们揭示了SGD培训的深神经网络的异常限制动态的机制来源。

在自然界中，对称治理规律，而对称打破纹理。在人工神经网络中，对称性是一种中央设计原则，可以在世界上有效地捕获规律，但对称性破裂的作用并不充分理解。在这里，我们开发了一个理论框架，用于研究神经网络中的“学习动态几何”，并揭示了现代神经网络效率和稳定性的明确对称性的关键机制。为了构建这种理解，我们使用连续时间拉格朗日制剂模拟梯度下降的离散学习动态，其中学习规则对应于动能，并且损耗函数对应于势能。然后，我们识别“动力学对称性破坏”（KSB），当动能明确地破坏潜在功能的对称性时的条件。我们概括了物理中已知的定理，以考虑KSB，并导致Noether费用的结果：“Noether的学习动态”（NLD）。最后，我们将NLD应用于具有归一化层的神经网络，并揭示了KSB如何引入“隐式自适应优化”的机制，建立由归一化层和RMSProp引起的学习动态之间的类比。总体而言，通过拉格朗日力学的镜头，我们建立了一个理论基础，以发现神经网络的学习动态的几何设计原则。

When simulating soft robots, both their morphology and their controllers play important roles in task performance. This paper introduces a new method to co-evolve these two components in the same process. We do that by using the hyperNEAT algorithm to generate two separate neural networks in one pass, one responsible for the design of the robot body structure and the other for the control of the robot. The key difference between our method and most existing approaches is that it does not treat the development of the morphology and the controller as separate processes. Similar to nature, our method derives both the "brain" and the "body" of an agent from a single genome and develops them together. While our approach is more realistic and doesn't require an arbitrary separation of processes during evolution, it also makes the problem more complex because the search space for this single genome becomes larger and any mutation to the genome affects "brain" and the "body" at the same time. Additionally, we present a new speciation function that takes into consideration both the genotypic distance, as is the standard for NEAT, and the similarity between robot bodies. By using this function, agents with very different bodies are more likely to be in different species, this allows robots with different morphologies to have more specialized controllers since they won't crossover with other robots that are too different from them. We evaluate the presented methods on four tasks and observe that even if the search space was larger, having a single genome makes the evolution process converge faster when compared to having separated genomes for body and control. The agents in our population also show morphologies with a high degree of regularity and controllers capable of coordinating the voxels to produce the necessary movements.

Taking into account background knowledge as the context has always been an important part of solving tasks that involve natural language. One representative example of such tasks is text-based games, where players need to make decisions based on both description text previously shown in the game, and their own background knowledge about the language and common sense. In this work, we investigate not simply giving common sense, as can be seen in prior research, but also its effective usage. We assume that a part of the environment states different from common sense should constitute one of the grounds for action selection. We propose a novel agent, DiffG-RL, which constructs a Difference Graph that organizes the environment states and common sense by means of interactive objects with a dedicated graph encoder. DiffG-RL also contains a framework for extracting the appropriate amount and representation of common sense from the source to support the construction of the graph. We validate DiffG-RL in experiments with text-based games that require common sense and show that it outperforms baselines by 17% of scores. The code is available at https://github.com/ibm/diffg-rl

Our team, Hibikino-Musashi@Home (the shortened name is HMA), was founded in 2010. It is based in the Kitakyushu Science and Research Park, Japan. We have participated in the RoboCup@Home Japan open competition open platform league every year since 2010. Moreover, we participated in the RoboCup 2017 Nagoya as open platform league and domestic standard platform league teams. Currently, the Hibikino-Musashi@Home team has 20 members from seven different laboratories based in the Kyushu Institute of Technology. In this paper, we introduce the activities of our team and the technologies.

本文档描述了Spotify出于学术研究目的发布的葡萄牙语播客数据集。我们概述了如何采样数据，有关集合的一些基本统计数据，以及有关巴西和葡萄牙方言的分发信息的简要信息。

深度神经网络（DNN）众所周知，很容易受到对抗例子的影响（AES）。此外，AE具有对抗性可传递性，这意味着为源模型生成的AE可以以非平凡的概率欺骗另一个黑框模型（目标模型）。在本文中，我们首次研究了包括Convmixer在内的模型之间的对抗性转移性的属性。为了客观地验证可转让性的属性，使用称为AutoAttack的基准攻击方法评估模型的鲁棒性。在图像分类实验中，Convmixer被确认对对抗性转移性较弱。

场景中光的极化信息对于各种图像处理和计算机视觉任务很有价值。平面偏光仪是一种有前途的方法，可以一次性地捕获不同方向的极化图像，而它需要颜色极化的表现。在本文中，我们提出了一个两步的颜色偏振化学网络〜（TCPDNET），该网络由两个颜色的表演和极化演示组成。我们还引入了YCBCR颜色空间中的重建损失，以提高TCPDNET的性能。实验比较表明，TCPDNET在极化图像的图像质量和Stokes参数的准确性方面优于现有方法。

实现接近真实机器人的高度准确的运动学或模拟器模型可以促进基于模型的控制（例如，模型预测性控制或线性质量调节器），基于模型的轨迹计划（例如，轨迹优化），并减少增强学习方法所需的学习时间。因此，这项工作的目的是学习运动学和/或模拟器模型与真实机器人之间的残余误差。这是使用自动调节和神经网络实现的，其中使用自动调整方法更新神经网络的参数，该方法应用了从无味的Kalman滤波器（UKF）公式进行方程式。使用此方法，我们仅使用少量数据对这些残差错误进行建模 - 当我们直接从硬件操作中学习改善模拟器/运动学模型时，这是必要的。我们演示了关于机器人硬件（例如操纵器组）的方法，并表明，通过学习的残差错误，我们可以进一步缩小运动学模型，模拟和真实机器人之间的现实差距。

深度神经网络（DNN）众所周知，很容易受到对抗例子的影响（AES）。此外，AE具有对抗性转移性，即为源模型傻瓜（目标）模型生成的AE。在本文中，我们首次研究了为对抗性强大防御的模型的可传递性。为了客观地验证可转让性的属性，使用称为AutoAttack的基准攻击方法评估模型的鲁棒性。在图像分类实验中，使用加密模型的使用不仅是对AE的鲁棒性，而且还可以减少AES在模型的可传递性方面的影响。